Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable host and cloud metadata processors by default #8596

Merged
merged 1 commit into from
Oct 17, 2018

Conversation

exekias
Copy link
Contributor

@exekias exekias commented Oct 7, 2018

InfraOps UI will leverage this info to provide better insights from the
application, we decided to enable these by default as that should
benefit most users.

@exekias exekias added in progress Pull request is currently in progress. libbeat needs_backport PR is waiting to be backported to other branches. v6.5.0 labels Oct 7, 2018
@exekias exekias force-pushed the enable-default-processors branch from b703a99 to b3197db Compare October 7, 2018 21:42
@ruflin
Copy link
Contributor

ruflin commented Oct 8, 2018

+1 on this change. What I wonder is what the effect on data size with this could be?

@willemdh
Copy link

willemdh commented Oct 9, 2018

@ruflin @exekias I know the add_host_metadata processor causes issues on Windows 2003 and 32-bit 2008 SP2 servers. When enabled, the Beats agent doesn't start. If I understand correctly this PR will enable the add_host_metadata processor by default? How will this PR work out on Windows 2003 and 32-bit 2008 SP2 servers?

@willemdh
Copy link

willemdh commented Oct 9, 2018

Hmm, seems that since 6.4.x Beats are no longer supported on Windows 2003 servers.. (tried updating today, the Beats services keep stopping..). Imho this really is a pity, as we have several 2003 servers left that are impossible to update due to all kinds of reasons (on which I have no control)....

@exekias
Copy link
Contributor Author

exekias commented Oct 12, 2018

hi @willemdh, this change only affects default configuration, you can always disable the processor if you don't really want it

@andrewkroh
Copy link
Member

I know the add_host_metadata processor causes issues on Windows 2003 and 32-bit 2008 SP2 servers. When enabled, the Beats agent doesn't start.

@willemdh I don't recall seeing any other bug reports for this. It would be good if you could open an issue for this an attach logs. Perhaps its something we can fix.

@exekias
Copy link
Contributor Author

exekias commented Oct 15, 2018

I've run Metricbeat both with and without add_host_metadata for a single host, several hours:

yellow open   metricbeat-6.4.2-2018.10.15      p2LP4anUSmmkJRrTYH8wHA   1   1      22161            0        7mb            7mb
yellow open   host-metricbeat-6.4.2-2018.10.15 Rd_nt2kjR7mbU5bGPcxBpQ   1   1      22159            0      7.3mb          7.3mb

The results are looking promising, WDYT?

@ruflin
Copy link
Contributor

ruflin commented Oct 16, 2018

@exekias Agree, very promising. Seems like compression works well on this one.

InfraOps UI will leverage this info to provide better insights from the
application, we decided to enable these by default as that should
benefit most users.
@exekias exekias force-pushed the enable-default-processors branch from b3197db to 1fa5275 Compare October 16, 2018 08:46
@exekias
Copy link
Contributor Author

exekias commented Oct 16, 2018

rebased with master, let me know if you want some more testing before going ahead

Copy link
Contributor

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now looking at the change again, it's not really that it's on my default. If someone doesn't have any processors configure, they are not enabled. It's only enabled if you use our short config. I'm wondering if we should also enable it on the code side by default? If we do that, the problem is that I think at the moment there is no such thing as add_host_metata.enabled: false that a user could use.

@exekias
Copy link
Contributor Author

exekias commented Oct 17, 2018

Yes, for now we are only talking about enabling it in the default configuration, with an easy way to opt out. Enabling them by code in a hardcoded way would be different, I would say we need to think more about that before moving that way.

@exekias exekias merged commit 9812157 into elastic:master Oct 17, 2018
@ruflin
Copy link
Contributor

ruflin commented Oct 17, 2018

I wonder if we should modify the changelog to "enabled by default in the config" to make it more obvious. Someone upgrading and keep his config will not see any change.

exekias added a commit to exekias/beats that referenced this pull request Oct 23, 2018
InfraOps UI will leverage this info to provide better insights from the
application, we decided to enable these by default as that should
benefit most users.

(cherry picked from commit 9812157)
@exekias exekias removed the needs_backport PR is waiting to be backported to other branches. label Oct 23, 2018
exekias added a commit that referenced this pull request Oct 23, 2018
…rs by default (#8686)

* Enable `host` and `cloud` metadata processors by default (#8596)

InfraOps UI will leverage this info to provide better insights from the
application, we decided to enable these by default as that should
benefit most users.

(cherry picked from commit 9812157)

* Update CHANGELOG.asciidoc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in progress Pull request is currently in progress. libbeat v6.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants